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This  report  describes  the  four  steps  used  to  determine  the  best  equipment  to 
use  for  collecting  soldier  performance  data.  Special  attention  was  paid  to  field 
data  collection,  where  operational  requirements  often  restrict  the  data  collection 
capability.  The  four  steps  involved  (a)  conducting  structured  interviews  with  ex¬ 
perienced  research  persons,  (b)  analyzing  questionnaires  mailed  to  researchers, 

(c)  preparing  a  list  of  the  data  collection  instrumentation  identified,  and  (d)  lo¬ 
cating  sources  for  the  equipment  items  and  doing  trade-off  analyses  to  identify 
the  most  cost-effective  supplier.  The  equipment  needed  fell  into  six  classes: 

(a)  audio,  (b)  video,  (c)  support  and  selected  supplementary,  (d)  environment 
and  weather,  (e)  work  space,  and  (f)  psychological  and  physiological.  This 
report  does  not  include  the  equipment  specification,  source,  and  cost  data  because 
such  data  will  become  obsolete  as  models  and  prices  change. 
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INSTRUMENTATION  REQUIREMENTS  FOR  COLLECTING 
SOLDIER  PERFORMANCE  DATA 

The  Army  Research  Institute  (ARI)  Fort  Sill  Field  Unit  (FSFU)  is 
involved  in  a  project  to  collect  data  on  soldier  errors  in 
automated  systems.  The  data  are  being  collected  as  part  of  a 
larger  ARI  research  effort  that  includes  the  creation  and 
validation  of  analytic  models  for  use  in  the  development  of 
weapon  systems.  This  larger  effort,  generally  referred  to  under 
the  rubric  "MANPRINT",  requires  that  analytic  techniques  that 
will  enable  materiel  developers  to  assess  the  impact  of  the 
soldiers  in  the  system  be  supported  by  a  data  base  that 
reliably  depicts  the  potential  performance  of  soldiers  using 
automated  systems. 

Information — data — concerning  soldier  error,  and  analytic 
techniques  to  manipulate  these  data,  are  needed  by  system 
designers  for  a  variety  of  reasons.  System  designers  require 
this  information  in  order  to  design  systems  that  reduce  the 
potential  for  error  rather  than  increase  it;  they  need  soldier 
performance  data  in  order  to  improve  the  maintainability  of 
weapon  systems  and  the  effectiveness  of  maintenance  procedures; 
they  need  to  know  what  errors  soldiers  may  make  in  order  to 
develop  training  equipment  and  training  programs;  and  they  need 
to  know  how  soldier  errors  are  likely  to  impact  weapon  system 
performance  in  combat  situations  so  that  they  can  reduce  the 
potential  of  soldier  error  affecting  the  reliability  of  the 
weapon  system. 

The  process  of  collecting  soldier  error  data  for  automated 
systems  permits,  even  requires,  the  data  to  be  collected  in  a 
variety  of  situations.  Some  data  can  be  acquired  in  a  laboratory 
setting;  some  requires  a  research  presence  in  the  classroom;  in 
some  cases  it  will  be  necessary  to  instrument  weapon  systems  in 
the  field  under  conditions  that  simulate — or  are  at  least 
representative — of  actual  battle  conditions.  The  purpose  of  the 
study  reported  in  this  document  was  to  determine  what  data 
collection  equipment  should  be  available  in  order  for  the  FSFU  to 
be  able  to  respond  effectively  to  opportunities  to  collect 
soldier  performance  data. 

Because  the  collection  of  data  in  the  field  is  difficult,  and  the 
field  is  likely  to  be  the  place  where  constraints  are  most 
severe,  this  study  had  as  its  primary  emphasis  the  collection  of 
data  in  situations  where  the  weapon  systems  were  in  use,  or  being 
tested,  in  operational  conditions.  The  goal  of  the  study 
reported  here  was  (a)  to  identify  those  items  of  equipment  that 
behavioral  scientists  would  need  to  collect  data  in  the  field; 

(b)  to  put  the  items  into  a  reasonable  order  of  priority;  (c)  to 
identify  sources  and  price  information  for  the  equipment  deemed 
necessary,  or  at  least  desirable. 
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A  previous  study  by  Duchein  and  Scott  (1987)  addressed  some  of 
these  same  issues.  In  their  report  the  authors  noted  that 
soldier  performance  is  now  recognized  as  a  vital  component  of 
weapon  system  effectiveness,  and  that  Army  Regulation  602-2 
requires  measurement  of  soldier  performance  of  critical 
operations  and  maintenance  tasks.  They  also  noted  that  the 
instrumentation  traditionally  used  to  record  such  performance 
tends  to  be  obvious  and  may  in  fact  intrude  on  the  simulated 
tactical  play  of  most  operational  tests.  Their  report  presented 
the  results  of  a  survey  which  identified  electronic 
instrumentation  which  is  suitable  for  detecting  and  measuring 
soldier  performance.  The  instrumentation  must  also  be 
sufficiently  unobtrusive  as  to  not  distract  soldiers  from  freely 
attending  to  the  tactical  situation  of  the  test.  The  four 
appendixes  to  the  Duchein  and  Scott  report  contain 
instrumentation  listings  with  summary  technical  specifications, 
American  vendor  sources,  and  authors'  comments. 

The  study  reported  here  went  beyond  the  Duchein  and  Scott  (1989) 
work  (which  focused  on  miniaturized  video  cameras,  video  and 
audio  recorders,  pan  and  tilt  control  units,  and  database 
recorders)  and  addressed  a  variety  of  electronic  and  other  types 
of  instruments.  The  researchers  went  to  Army  test  organizations 
to  draw  upon  their  experience  in  order  to  obtain  data  to  identify 
and  prioritize  data  collection  equipment;  to  identify  sources;  to 
make  judgements — based  on  cost  and  utility — of  what  was  the  best 
item;  and  provide  current  cost  data.  This  report  does  not 
include  the  source-cost  type  data  because  such  data  is  subject  to 
somewhat  rapid  change,  and  is  likely  to  be  quite  obsolete  in  a 
relatively  short  time.  This  information  has  been  furnished 
separately  by  the  researchers,  however,  and  questions  concerning 
specific  items  will  be  answered,  and  appropriate  material 
furnished,  by  the  ARI  FSFU  upon  request  during  the  period  it  is 
considered  still  current. 


Approach 

This  study  involved  four  phases.  In  the  first  phase  structured 
interviews  were  conducted  at  selected  organizations  who  had 
extensive  experience  at  data  collection  in  the  field.  In  the 
second  phase  the  structuring  questionnaire  was  converted  to  a 
mailable  questionnaire  and  sent  to  additional  organizations  to 
obtain  a  broader  set  of  responses.  In  the  third  phase  a  list  of 
equipment,  and  data  collection  requirements,  was  prepared  which 
identified  the  data  collection— and  support — equipment  which 
researchers  felt  was  needed.  During  the  fourth  phase  sources 
were  found  for  the  various  equipment  items;  analyses  were 
performed  to  arrange  equipment  items  according  to  priority;  and 
cost  effectiveness  analyses  were  done  to  develop  recommendations 
for  specific  items. 
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This  report  describes  how  the  study  was  conducted  and  discusses 
the  equipment  requirements.  The  source  and  cost  data,  which  was 
provided  to  ARI,  is  not  included  here  because  such  information  is 
rather  ephemeral,  and  is  subject  to  rapid  obsolescence  as  models 
and  prices  change. 

Structured  Interviews 

The  Instrumentation  Questionnaire  developed  for  the  structured 
interview  phase  of  this  project  consisted  of  five  sections.  The 
first  section  asked  what  equipment  was  currently  used  to  collect 
data  on  human  performance.  Later  sections  asked  for 
physiological  instrumentation;  environment  data;  atmospheric 
condition  data;  and  anthropometric  and  workspace  data  collection 
equipment.  A  26  page  form,  a  sample  page  of  which  is  shown  in 
Figure  1,  was  used  to  guide,  and  record  data  from,  the  interview. 
At  each  location  the  interviewer  asked  the  subject  matter  expert 
(SME) ,  or  experts,  about  the  specific  items  of  equipment  which 
were  already  entered  in  the  form,  and  then  the  interviewer 
encouraged  the  SMEs  to  add  other  equipment  types  and  models  which 
they  had  used.  After  each  section  the  SMEs  were  also  asked  about 
equipment  that  could  be  useful  in  field  data  collection.  The 
questionnaire  was  used  in  the  conduct  of  personal  interviews  with 
personnel  at  five  ARI  field  units  (Ft.  Sill,  Ft.  Knox,  Ft. 

Rucker,  Ft.  Bliss,  and  Ft.  Hood)  and  five  TEXCOM  representatives 
(Aviation  Test  Board,  Armor  and  Engineering  Board,  Air  Defense 
Board,  Field  Artillery  Board,  and  the  Instrumentation  Directorate 
of  TEXCOM) . 

Mail  Survey 

Following  the  completion  of  the  personal  interviews,  a  second 
survey  instrument  was  developed  for  use  in  a  mail  solicitation. 

A  sample  of  the  revised  form  is  shown  in  Figure  2.  It  included 
specific  equipment  items  and  space  to  add  items,  as  with  the 
first  form.  The  respondents  were  asked  to  indicate  if  they  had 
in  fact  used  specific  items  of  equipment,  and  also  to  list  items 
they  thought  would  likely  be  needed.  The  mail  survey  was  sent  to 
eleven  ARI  field  units  (Ft.  Knox;  Ft.  Rucker;  Ft.  Hood;  Ft. 
Leavenworth;  Ft.  Benning;  Boise,  Idaho;  St.  Louis,  Missouri; 
Orlando,  Florida;  Ft.  Huachuca;  Monterey,  California;  and  Ft. 
Bliss)  to  determine  their  instrumentation  needs.  Of  the  eleven 
mail  surveys  sent  out,  eight  were  completed  and  returned  (Ft. 
Knox;  Ft.  Rucker;  Ft.  Hood;  Ft.  Leavenworth;  Ft.  Benning;  Boise, 
Idaho;  St.  Louis,  Missouri;  and  Orlando,  Florida) . 

Equipment  Requirements 

The  results  of  the  interviews  and  the  surveys  were  then  used  to 
develop  a  list  of  the  basic  instrumentation  and  ancillary 
equipment  identified  by  the  SMEs.  As  the  data  were  being 
collated  it  became  apparent  that  the  five  categories  (human 
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QUESTIONNAIRE 


1.  What  equipment  are  you  currently  using  to  collect  human 
performance  data?  (Please  place  a  check  mark  next  to  the  answer (s) 
that  is/are  applicable) . 

None _  (if  you  checked  None  go  to  next  question) 

Human  Observer (s)  _ 


Equipment 

Type 

In  Use 
Yes  No 

Description  (Make  &  Capability 

Quant 

Videotape- 

Recording 

System 

Audio  Tape 
Recorder 

Multiple 

Event 

Counter 

Figure  1.  Specimen  page  from  structured  interview  questionnaire. 
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QUESTIONNAIRE 


VIDEO  DATA  COLLECTION 

1.  Of  the  video  equipment  listed  below,  which  have  you  used  to 
collect  human  performance  data,  and  which  equipment  would  you  use 
if  it  were  available  to  you? 


Please  place 

Equipment 

Tvpe 

a  check  mark 

Have  Used 

where 

Would 

applicable. 

Use  COMMENTS 

YES 

I  NO 

YES  I 

NO  (INCL.  TYPE  OF  DATA  COLLECTED) 

Micro-video- 

1 

1  1 

1 

1 

camera  svs. 

1 

1  1 

1 

1 

-monochrome 

1 

1  1 

1 

1 

-low  light 

1 

1  1 

1 

1 

level  cap 

| 

;  i 

1 

| 

Mini-video- 

1  i 

i 

i 

camera  svs. 

1 

i  i 

i 

i 

-color 

1 

i  i 

i 

i 

-high  resol. 

1 

;  ; 

i 

| 

Video  Cam- 

1 

i  i 

i 

i 

corder 

1 

i  i 

i 

i 

-w/audio  cap. 

1 

i  i 

i 

| 

Time  code 

I 

i  i 

i 

i 

SWQWtgr 

1 

i  i 

i 

-for  video 

! 

i  i 

i 

i 

cameras 

1 

i  i 

i 

i 

— . - 

LIST  BELOW  ANY  VIDEO  EQUIPMENT  THAT  YOU  HAVE  USED  THAT  IS  NOT 
LISTED  ABOVE: 


Figure  2.  Specimen  page  from  revised  questionnaire  used  for 
mail  survey. 
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performance;  physiological;  environmental;  atmospheric;  and 
anthropometric  and  workspace)  used  in  the  interview  and  mail  data 
collection  process  were  not  suitable  for  presenting  the  actual 
equipment  listings.  Therefore,  a  new  set  of  categories  were 
developed.  The  new  classification  system  reflects  the  fact  that 
SMEs  see  a  need  to  collect  both  audio  and  video  data  in  order  to 
determine  soldier  performance;  that  in  order  to  meaningfully 
analyze  these  data,  information  concerning  the  conditions  under 
which  the  soldiers  performed  is  needed;  and  that  some  equipment 
is  needed  to  support  the  data  collection  process,  and  the  data 
collection  team  itself.  It  appeared  that  six  kinds  of  equipment 
were  required:  (a)  audio  equipment;  (b)  video  equipment;  (c) 
support  equipment;  and  three  kinds  of  supplementary  equipment 
(d)  psychological  and  physiological;  (e)  environment  and  weather; 
(f)  workspace  and  anthropometric. 

Since  part  of  the  project  involved  the  prioritizing  of  the 
equipment  items  nominated  by  the  various  SMEs,  some  of  the  item 
sets  were  further  divided  intc  primary  and  secondary  lists  which 
reflect  the  need-to-have  versus  nice-to-have  division,  which 
permeates  the  opinions  of  SMEs  who  have  always  had  to  deal  with 
both  the  constraints  of  the  field  environment  and  the  needs  of 
research. 

Audio  Equipment. 

The  audio  equipment  identified  by  the  SMEs  included:  directional 
microphones,  voice  activated  microphones,  head  sets,  and  tape 
recording  systems  for  use  with  these  systems.  Implicit  in  the 
SME  identifications  was  the  realization  that  the  audio  collection 
process  would  often  require  individually  mounted  radio 
transmitters  which  would  make  it  possible  to  instrument  soldiers 
in  work  locations  where  space  was  at  a  premium  but  do  the 
recording  elsewhere. 

Video  Equipment. 

SMEs  felt  that  both  micro  and  mini  video  cameras  would  be  needed, 
and  that  video  camcorders  should  also  be  available  for  use  when 
space  conditions  permitted.  Recording  systems  for  the  video 
cameras  were  also  specified,  along  with  time-code  generators  and 
suitable  editing  systems.  Camera  mounting  fixtures  and  tripods 
were  considered  nice-to-have  items.  SMEs  apparently  believed 
that  the  camera  mounting  requirement  was  often  so  unique  that 
unless  custom  built  fixtures  were  created,  duct  tape  or  velcro 
would  need  to  be  used. 

Support  Equipment. 

The  field  research  experienced  SMEs  who  were  interviewed,  or  who 
responded  to  the  mail  survey,  were  quick  to  point  out  that  an 
important  consideration  in  field  data  collection  is  the  need  to 
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provide  support  for  the  data  collection  process  itself.  One, 
oft.  n  overlooked,  requirement  that  is  of  overwhelming  importance 
to  the  success  of  a  field  data  collection  effort  is  the 
requirement  to  move  data  collection  equipment  to  and  from  the 
field  site.  Many  SMEs  noted  that  suitable  shipping  containers 
are  an  important  support  requirement.  They  felt  that  all 
equipment  which  may  be  taken  to  the  field  should  have  suitable, 
rugged  containers.  The  containers  need  to  be  types  which  can  be 
shipped  by  commercial  or  military  air  or  standard  surface 
shipping  methods.  To  the  extent  possible  equipment  should  be 
shipped  in  containers  small  enough  to  meet  the  size  requirement 
of  United  Parcel  Service  (UPS) ,  the  United  States  Post  Office  or 
even  to  carry  on  commercial  aircraft.  When  items  are  in  cases 
that  are  large  or  heavy  the  cases  should  have  handles  and  wheels 
to  make  transporting  them  easy  for  individuals. 

The  need  for  computer  support  for  research  teams  in  field 
locations  was  also  noted.  Often  field  researchers  must  work  out 
of  motels,  or  from  relatively  unfurnished  office  space  at  or  near 
the  field  research  location.  Potential  computer  requirements  for 
the  test  site  and  the  research  base  include  laptop  computers  for 
field  site  use;  a  transportable  ruggedized  PC  for  base 
operations;  a  modem  for  communicating  "back  home";  and 
appropriate  software.  Software  that  has  proven  useful  includes 
word  processing,  spreadsheet,  statistical  and  communication 
programs.  Some  researchers  felt  that  the  relatively  new 
electronic  clipboard  technology  held  promise,  and  recommended 
such  systems,  with  the  appropriate  download  capability, as  a  good 
extension  of  the  data  collection  process. 

Communications  among  researchers  was  often  seen  as  a  requirement 
and  most  SMEs  felt  that  handheld  CB  radios,  and  a  CB  base  station 
should  be  available  for  use  in  the  field  if  needed.  It  was  also 
suggested  that  since  cellular  phone  service  is  becoming  so  common 
a  good  equipment  suite  should  include  the  phone  sets  and  service 
subscriptions  needed  to  operate  in  the  field  if  the  service  is 
available.  Although  it  is  not  an  equipment  item  it  is  worth 
noting  that  a  research  group  that  takes  radio  operated  data 
collection  equipment,  CB  radios,  or  cellular  phones  to  an 
operational  unit  will  need  to  make  the  necessary  checks,  and  get 
the  necessary  approvals,  to  assure  that  the  frequencies  they  use 
do  nor  int  '  v? th  unit  operations — or  the  local  TV  station — 

and  that  no  ocar. 

Of  course  with  so  much  battery  operated  equipment,  and  the 
potential  for  having  to  supply  the  research  teams  110  volt  ac 
requirement  the  need  sometimes  arises  to  provide  the  research 
teams  with  power  support.  Thus,  a  small  generator;  rechargeable 
batteries,  in  all  sizes;  and  battery  rechargers  are  required. 
Other  nice-to-have  items  include  35mm  cameras  and  Polaroid 
cameras  to  obtain  site  views  and  sometimes  data;  and  night  vision 
goggles  for  those  occasions  when  the  researchers  need  to  operate 
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during  blackout  conditions.  And,  of  course,  there  is  a  final 
essential  support  requirement:  there  must  be  a  tool  kit  to  help 
with  wiring  and  make  minor  repairs. 

Supplementary  Ecruipment  f Environmental  and  Weather  Related) . 

SMEs  indicated  that  the  sound  level  and  illumination  level  were 
important  data  items  in  many  tests;  and  that  a  gas  analyzer  could 
often  be  used  to  obtain  important  information  about  the  soldiers 
immediate  work  areas.  It  was  also  clear  that  for  testing  in  the 
field  a  portable  set  of  weather  measuring  devices  (temperature, 
humidity,  and  wind)  would  often  be  needed  to  measure  both  the 
actual  work  area  conditions  and  the  general  conditions  at  the 
field  site.  Vibration  level  measuring  equipment,  an 
accelerometer,  and  an  air-flow  meter  were  identified  as  of 
secondary  importance  but  occasionally  required. 

Supplementary  Equipment  (Psychological  and  Physiological) . 

A  digital  thermometer  for  body  temperature;  a  general  purpose 
vision  test  device  to  obtain  acuity,  color-vision,  and 
stereoscopic  measures;  and  a  stop  watch  to  obtain  pulse  rate  and 
time  other  events,  were  considered  essential.  A  reaction  time 
measuring  device  and  a  blood  pressure  monitor  were  considered  of 
secondary  importance. 

Supplementary  Equipment  (Workspace  related). 

The  only  piece  of  equipment  that  was  considered  of  primary 
importance  was  a  good  tape  measure  (25'  perhaps)  which  would  be 
used  to  measure  distances,  room  and  area  sizes,  and  space  access 
restrictions.  A  scale,  a  set  of  anthropometric  measuring  tools, 
a  torque  meter,  and  a  force  gauge  were  considered  nice  to  have 
for  occasional  use. 

Trade  off  Analyses.  Cost  and  Sources. 

As  a  part  of  this  study  the  equipment  items  identified  were 
subjected  to  further  analyses  to  provide  data  of  value  to  persons 
actually  needing  to  order  equipment  for  field  data  collection. 
There  were  three  steps  to  this  process:  (a)  creating  specific 
statements — functions  and  specifications — for  the  needed  items 
(b)  conducting  a  market  survey  to  identify  sources;  and  (c)  doing 
a  trade-off  analysis  in  order  to  recommend  what  appeared  to  be 
the  most  cost  effective  source. 

For  each  specific  type  of  instrumentation  or  ancillary  equipment 
requirement  identified,  functional  and  technical  descriptions, 
and  specifications  were  prepared.  The  descriptions  were  prepared 
to  a  level  consistent  with  the  'evel  of  detail  required  to 
initiate  a  government  procurement  action.  The  functions  and 
specifications  were  obtained  from  persons  who  had  ordered  such 
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equipment,  by  personal  contact  with  suppliers,  and  from 
catalogues. 

After  obtaining  a  suitable,  specific  item  description  the 
researchers  identified  three  sources  for  equipment  items  that  met 
the  specification.  When  three  sources  were  not  available  it  was 
so  noted  and  the  reason  recorded.  Cost  data  was  then  obtained. 
The  cost  data  included:  item  cost;  duration  of  initial  warranty; 
cost  of  maintenance  contract,  if  available,  on  items  exceeding 
$1000;  and  shipping  or  set  up  costs  if  required. 

These  cost  data;  the  SME  "need"  and  "could  use"  data;  the 
specification;  and  the  source  and  cost  data  were  then  used  to 
perform  a  trade-off  analysis  by  which  the  most  cost  effective 
equipment  items  were  identified.  The  results  of  the  trade-off 
analyses  were  then  collated  and  provided,  under  separate  cover, 
for  possible  use  in  ordering  equipment.  Because  the  specific 
cost,  source,  and  model  information  is  only  applicable  in  the 
period  when  the  analyses  were  run  they  are  not  included  in  this 
report.  However,  it  is  worth  noting  that  to  obtain  a  basic  field 
data  collection  capability,  a  laboratory  would  have  to  expend 
about  $75,000  to  $125,000  depending  on  how  many  data  collection 
points  it  was  necessary  to  instrument.  To  obtain  the  additional 
items  that  are  only  occasionally  of  value  an  additional  $25,000 
to  $50,000  would  need  to  be  spent. 
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